Method for early diagnosis of keratoconus based on multi-modal data

ABSTRACT

A method for early diagnosis of keratoconus based on multi-modal data considers a mutual relationship between both eyes using four refractive maps for corneas of the both eyes and absolute corneal elevation data, and combines the deep convolutional network method, the traditional support vector machine (SVM) method in machine learning, and the elevation map enhancement method with adjustable best-fit-sphere (BFS) to identify sensitivity and specificity of a focus and balance the sensitivity and specificity. With multi-dimensional comprehensive judgment of a keratoconus morbidity with a patient as a unit, combined with binocular data including both manual selection features and deep network learning from big data, the diagnosis method has higher robustness and accuracy.

CROSS REFERENCE TO THE RELATED APPLICATIONS

This application is the national phase entry of InternationalApplication No. PCT/CN2021/110812, filed on Aug. 5, 2021, which is basedupon and claims priority to Chinese Patent Application No.202110717940.2, filed on Jun. 28, 2021, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an ophthalmic data diagnosistechnology, and in particular, to a method for early diagnosis ofkeratoconus based on multi-modal data.

BACKGROUND

Keratoconus is a clinical ophthalmic disease characterized by cornealdilation, central thinning, forward protrusion, and conical shape. Suchdisease is a contraindication to refractive surgery, occurs in one orboth eyes, and usually results in a significant loss of vision. Thekeratoconus usually develops first from the posterior corneal surface,and then gradually develops to the anterior corneal surface.

At present, diagnosis and treatment of the keratoconus has developedinto a clinical disease with close cooperation of keratopathy,refractive surgery, and optometry. Usually, a method for diagnosing thekeratoconus based on corneal topography is a clinical statisticalmethod. Most of the morphological features of the corneal topographycombined with clinical parameters and medical history are used todiagnose and clinically stage the keratoconus. The result of astatistical model is a summary parameter, and a parameter boundary isobtained according to a known diagnostic result data set, such as thewidely used KISA index, inferior-superior (IS) index, and surfaceregularity index (SRI)\surface asymmetry index (SAI). Most of thesemethods are limited by platform data, overly rely on a limited number ofartificially identified monocular features, ignore the relationshipbetween both eyes, and fail to give a good judgment on early keratoconusand forme fruste keratoconus due to different sensitivity andspecificity of different indexes.

SUMMARY

Aiming at the problem of early diagnosis of keratoconus, a method forearly diagnosis of keratoconus (including subclinical asymptomatickeratoconus and forme fruste keratoconus) based on multi-modal data isprovided. Based on multi-modal refractive topographic matrix data, incombination with the neural network convolution method, the eigenvaluesupport vector machine (SVM) method, the binocular contrast method, andthe enhanced topography method with adjustable best-fit-sphere (BFS),comprehensive diagnosis results of the keratoconus are given, and themethod has more excellent robustness and accuracy especially forscreening and diagnosis of early posterior keratoconus and forme frustekeratoconus.

A technical solution of the present disclosure is as follows: a methodfor early diagnosis of keratoconus based on multi-modal dataspecifically includes the following steps:

1) acquiring binocular multi-modal data including four refractive mapsfor corneas of both eyes and absolute corneal elevation data, where thefour refractive maps for the corneas include an axial curvature map ofan anterior corneal surface, a relative elevation topography of theanterior corneal surface, a relative elevation topography of a posteriorcorneal surface, and a corneal thickness topography; and the absolutecorneal elevation data includes absolute elevation data of the anteriorand posterior corneal surfaces;

2) according to classified cases, associating the binocular multi-modaldata with classification categories, and classifying the data accordingto requirements;

3) unifying data of topographies and the elevation data of the anteriorand posterior corneal surfaces in the four refractive maps in step 2)into data matrices of a same size;

4) based on the above data, determining early keratoconus of the botheyes using four branch methods, which are respectively recorded as abranch A method, a branch B method, a branch C method, and a branch Dmethod, where

the branch A method is as follows: after data processing, sending all ofthe data matrices of the four refractive maps to a classificationnetwork of a deep convolutional network to identify sensitivity andspecificity of the keratoconus and obtain a classification result P(A)output for a certain case;

the branch B method is as follows: calculating eigenvalues of eachgraphic data matrix in all of the data matrices of the four refractivemaps, and sending eigenvalue data to a binary classification methodusing an SVM to identify the sensitivity and specificity of thekeratoconus and obtain a classification result P(B) output for thecertain case;

the branch C method is as follows: comparing the absolute elevation dataof the anterior and posterior corneal surfaces with BFS data to obtain acritical threshold between keratoconus cases and normal cases, so as todetermine a classification result P(C) output for the certain case; and

the branch D method is as follows: obtaining optimal sensitivity andspecificity as well as a probability P(D) of the keratoconus of thecertain case using the critical threshold or using an SVM classificationmethod by taking an average value, a maximum value, and a standarddeviation of the data matrices of the four refractive maps for left andright eyes as feature quantities; and

5) weighting and accumulating final results in the branch A, B, C, and Dmethods to obtain a final probability of the keratoconus of the certaincase.

Preferably, the branch A method may include the following specificsteps:

A-1: performing data scaling: scaling all of the data matrices of thefour refractive maps processed in step 3) to a size of 224×224 by linearinterpolation;

A-2: performing data normalization: dividing data in step A-1 into atraining set and a validation set according to a ratio of 7:3, thencalculating means and standard deviations of the data matrices of thefour refractive maps on the training set respectively to correspondinglyobtain 4 means and 4 standard deviations, and then normalizing datamatrices of four refractive maps for all cases with the means and thestandard deviations;

A-3: based on classification network design of the deep convolutionalnetwork, performing binary classification on the data matrices of thefour refractive maps using a Resnet50 classification network to identifya normal cornea and the keratoconus in one eye;

A-4: training a classification model: connecting the data matrices ofthe four refractive maps according to a channel to obtain an input of 4channels, where data amplification may use rotation, translation, andrandom fuzzy preprocessing, and a loss function may use a binary crossentropy function; using training weights of MobileNetV3 on an IMAGENETdataset as initial weights, and then performing fine-tuning training;and finally selecting a training weight with a smallest differencebetween loss values of the training set and the validation set as atraining result;

A-5: performing model index evaluation: making predictions on thevalidation set, and then comparing with a true value for evaluation tofinally obtain the sensitivity and specificity of the branch A method inidentifying the keratoconus; and

A-6: outputting results: if test sensitivity and specificity of thetraining set in step A-5 meet requirements, recording probabilities ofthe keratoconus of the branch A method for the certain case as p(Al) andp(Ar) respectively; and outputting classification results: P(A)=p(Al),(p(Al)>p(Ar)), and p(Ar), (p(Al)<p(Ar)).

Preferably, the branch B method may include the following specificsteps:

B-1: calculating axial curvature eigenvalues of the anterior cornealsurface: calculating a maximum curvature point and position coordinatesin a data matrix of an axial curvature of the anterior corneal surface,calculating an IS value of a difference between upper and lowerrefractive power at a position with a diameter of 6 mm, and calculatingan SRI and an SAI within a diameter of 4.5 mm;

B-2: calculating relative elevation eigenvalues of the anterior cornealsurface: calculating a maximum elevation and position coordinates in adata matrix of a relative elevation of the anterior corneal surface;

B-3: calculating relative elevation eigenvalues of the posterior cornealsurface: calculating a maximum elevation and position coordinates in adata matrix of a relative elevation of the posterior corneal surface;

B-4: calculating corneal thickness eigenvalues: calculating a minimumthickness and position coordinates in a data matrix of a cornealthickness, and calculating a thickness at a corneal vertex;

B-5: calculating distance eigenvalues: calculating a distance from aposition of the maximum elevation of the anterior corneal surface instep B-2 to a position of the maximum elevation of the posterior cornealsurface in step B-3, calculating a distance from the position of themaximum elevation of the anterior corneal surface in step B-2 to aposition of the minimum corneal thickness in step B-4, and calculating adistance from the position of the maximum elevation of the posteriorcorneal surface in step B-3 to the position of the minimum cornealthickness in step B-4;

B-6: calculating corneal volume eigenvalues: performing volume integralon the data matrix of the corneal thickness within a radius of 4.5 mm toobtain a corneal volume;

B-7: normalizing all of the eigenvalues in steps B-1 to B-6, anddividing all of the normalized case data eigenvalues into a training setand a validation set according to a ratio of 7:3;

B-8: performing feature training on feature data of the training setnormalized in step B-7 by the binary classification method using theSVM, where a radial basis function (RBF) kernel may be selected in theprocess, and optimal c and g are obtained to train data usingcross-validation and grid-search;

B-9: performing model index evaluation: making predictions on thevalidation set, and then comparing with a true value for evaluation tofinally obtain the sensitivity and specificity of the branch B method inidentifying the keratoconus; and

B-10: outputting results: if test sensitivity and specificity of thetraining set in step B-9 meet requirements, recording probabilities ofthe keratoconus of the branch B method for the certain case as p(Bl) andp(Br) respectively; and outputting classification results: P(B)=p(Bl),(p(Bl)>p(Br)), and p(Br), (p(Bl)<p(Br)).

Preferably, the branch C method may include the following specificsteps:

C-1: calculating standard relative elevation data of the anteriorcorneal surface: for the absolute elevation data of the anterior andposterior corneal surfaces, performing spherical fitting on the absoluteelevation data of the anterior corneal surface within a diameter of 8 mmto obtain a BFS value, and taking an elevation difference between thedata of the anterior corneal surface and the obtained BFS as thestandard relative elevation data of the anterior corneal surface;

C-2: calculating feature elevation data of the anterior corneal surface:for the absolute elevation data of the anterior and posterior cornealsurfaces, removing data within a radius of 2 mm of a thinnest point forspherical fitting by taking the absolute elevation data of the anteriorcorneal surface within a diameter of 8 mm as a benchmark to obtain a BFSvalue; and offsetting 5 groups of data up and down respectively bytaking the current BFS as a benchmark and 0.2 mm as a stride to obtain11 groups of different BFS values, and taking an elevation differencebetween the data of the anterior corneal surface and the obtaineddifferent BFS as the feature relative elevation data of the anteriorcorneal surface;

C-3: calculating enhanced elevation data of the anterior cornealsurface: calculating a difference between the standard relativeelevation data obtained in step C-1 and the 11 groups of featurerelative elevation data obtained in step C-2 to obtain 11 groups ofenhanced data of the anterior corneal surface;

C-4: calculating standard relative elevation data of the posteriorcorneal surface: for the absolute elevation data of the anterior andposterior corneal surfaces, performing spherical fitting on the absoluteelevation data of the posterior corneal surface within a diameter of 8mm to obtain a BFS value, and taking an elevation difference between thedata of the posterior corneal surface and the obtained BFS as thestandard relative elevation data of the posterior corneal surface;

C-5: calculating feature elevation data of the posterior cornealsurface: for the absolute elevation data of the anterior and posteriorcorneal surfaces, removing data within a radius of 2 mm of a thinnestpoint for spherical fitting by taking the absolute elevation data of theposterior corneal surface within a diameter of 8 mm as a benchmark toobtain a BFS value; and offsetting 5 groups of data up and downrespectively by taking the current BFS as a benchmark and 0.2 mm as astride to obtain 11 groups of different BFS values, and taking anelevation difference between the data of the posterior corneal surfaceand the obtained different BFS as the feature relative elevation data ofthe posterior corneal surface;

C-6: calculating enhanced elevation data of the posterior cornealsurface: calculating a difference between the standard relativeelevation data of the posterior corneal surface obtained in step C-4 andthe 11 groups of feature relative elevation data obtained in step C-5 toobtain 11 groups of enhanced data of the posterior corneal surface;

C-7: counting a critical threshold between the keratoconus cases and thenormal cases in each group of data in combination with all sample databy taking a matrix of a total of 22 groups of enhanced data of theanterior and posterior corneal surfaces obtained in steps C-3 and C-6 asa feature; and

C-8: recording probabilities of the keratoconus of the branch C methodfor the certain case as p(Cl) and p(Cr) respectively, where p(Cl) andp(Cr) may be obtained through accumulation by taking a differencebetween each group of enhanced data calculated from a current case andthe critical threshold obtained in step C-7 as a weight ratio; andoutputting classification results: P(C)=p(Cl), (p(Cl)>p(Cr)), and p(Cr),(p(Cl)<p(Cr)).

Preferably, the branch D method may include the following specificsteps:

D-1: unifying data orientations: mirroring the data matrices of the fourrefractive maps for the right eye in a longitudinal direction, andunifying nasal and bitamporal orientations of the data matrices of thefour refractive maps for the left and right eyes;

D-2: obtaining diff diagram matrices of the four refractive maps:calculating a point-to-point difference of the data matrices of the fourrefractive maps for the left and right eyes respectively and then takingan absolute value to obtain the diff diagram data matrices of the fourrefractive maps;

D-3: calculating diff data features: calculating an average value, amaximum value, and a standard deviation of all data in the diff diagramdata matrices of the four refractive maps within a diameter of 6 mmrespectively as feature quantities;

D-4: counting a critical threshold between the keratoconus cases and thenormal cases in each group of data in combination with all sample databy taking the 12 groups of average values, maximum values, and standarddeviations of diff diagrams of the four refractive maps for the corneasof the left and right eyes obtained in step D-3 as features, ornormalizing features of all types of diff data for training and testingusing the SVM classification method to give the optimal sensitivity andspecificity; and

D-5: recording a probability of the keratoconus of the branch D methodfor the certain case as P(D), where P(D) may be obtained throughaccumulation by taking a difference between an eigenvalue of each groupof diff data calculated from a current case and the critical thresholdobtained in step D-4 as a weight ratio.

The present disclosure has the following beneficial effects: the methodfor early diagnosis of keratoconus based on multi-modal data of thepresent disclosure considers a mutual relationship between both eyes,and combines the deep convolutional network method, the traditional SVMmethod in machine learning, and the elevation map enhancement methodwith adjustable BFS to identify sensitivity and specificity of a focusand balance the sensitivity and specificity. With multi-dimensionalcomprehensive judgment of a keratoconus morbidity with a patient as aunit, combined with binocular data including both manual selectionfeatures and deep network learning from big data, the diagnosis methodhas higher robustness and accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of a method for early diagnosis of keratoconusbased on multi-modal data of the present disclosure;

FIG. 2 shows four refractive maps for corneas;

FIG. 3 shows an absolute elevation map of anterior and posterior cornealsurfaces;

FIG. 4 is a structural diagram of a classification network in the methodof the present disclosure; and

FIG. 5 shows Diff diagrams of four refractive maps for corneas of leftand right eyes.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The present disclosure will be described in detail in conjunction withthe accompanying drawings and specific embodiments. The embodiments areimplemented on the premise of the technical solutions of the presentdisclosure. The following presents detailed implementations and specificoperation processes. The protection scope of the present disclosure,however, is not limited to the following embodiments.

As shown in FIG. 1 , a flow chart of a method for early diagnosis ofkeratoconus based on multi-modal data specifically includes thefollowing steps.

1. Binocular multi-modal data is acquired, including four refractivemaps for corneas of both eyes and absolute corneal elevation data. Thefour refractive maps are obtained through a three-dimensional anteriorsegment analyzer or an anterior segment optical coherence tomography(OCT) measuring device. As shown in FIG. 2 , the four refractive mapsfor the corneas include an axial curvature map of an anterior cornealsurface, a relative elevation topography of the anterior cornealsurface, a relative elevation topography of a posterior corneal surface,and a corneal thickness topography. FIG. 3 shows absolute elevation dataof the anterior and posterior corneal surfaces.

2. According to classified cases, the binocular multi-modal data isassociated with classification categories. The number of classificationsdepends on specific needs, such as binary classification as keratoconusand a normal cornea.

3. According to the four refractive maps for the corneas and the colorcorresponding code, the two-dimensional full sampling data matrix withina diameter of 9 mm of the topography is obtained with a data stride of0.02 mm (the four maps in the four refractive maps are calledtopographies). The size of data of each topography and the elevationdata matrices of the anterior and posterior corneal surfaces is 451×451.

4. Based on the above data, early keratoconus of the both eyes isdetermined using four branch methods, which are respectively recorded asa branch A method, a branch B method, a branch C method, and a branch Dmethod.

The branch A method is as follows: this branch method takes a singlecase as a reference. Each data in all of the data matrices of the fourrefractive maps is involved in the calculation. All the factors that mayaffect the keratoconus judgment are retained as much as possible. Thepurpose of judging lesions in massive data is achieved through machineself-learning, which can be independent of the selection of humansubjective features to enhance the specificity of identifying the focus.

A-1: Data scaling is performed: all of the data matrices of the fourrefractive maps processed in step 3 is scaled to a size of 224×224 bylinear interpolation.

A-2: Data normalization is performed: data in step A-1 is divided into atraining set and a validation set according to a ratio of 7:3. Thenmeans and standard deviations of the data matrices of the fourrefractive maps are calculated on the training set respectively tocorrespondingly obtain 4 means and 4 standard deviations. Then datamatrices of four refractive maps for all cases are normalized with themeans and the standard deviations.

A-3: Based on classification network design of the deep convolutionalnetwork, binary classification is performed on the data matrices of thefour refractive maps using a Resnet50 classification network to identifya normal cornea and the keratoconus in one eye. For the built Resnet50classification network model, the backbone network remains unchanged,only the channel input of the first convolutional layer is modified to4, and the output number of the finally output fully connected layer is2. The network structure is shown in FIG. 4 .

A-4: A classification model is trained: the data matrices of the fourrefractive maps are connected according to a channel to obtain an inputof 4 channels. Data amplification uses preprocessing such as rotation,translation, and random fuzziness. A loss function uses a binary crossentropy function. Training weights of MobileNetV3 on an IMAGENET datasetare used as initial weights, and then fine-tuning training is performed.Iterative training is performed for 60 epoches with an initial learningrate of 0.01, and the learning rate is reduced by 10 times in 20 epochesand 40 epoches respectively. A training weight with a smallestdifference between loss values of the training set and the validationset is finally selected as a training result.

A-5: Model index evaluation is performed: predictions are made on thevalidation set, and then compared with a true value for evaluation tofinally obtain the sensitivity and specificity of the branch A method inidentifying the keratoconus.

A-6: Results are output: if test sensitivity and specificity of thetraining set in step A-5 meet requirements, probabilities of thekeratoconus of the branch A method for a certain case are recorded asp(Al) and p(Ar) respectively. Classification results are output:P(A)=p(Al), (p(Al)>p(Ar)), and p(Ar), (p(Al)<p(Ar)).

The branch B method is as follows: this branch method takes a singlecase as a reference. The features that can directly reflect the lesionare manually defined. Then the supervised learning method SVM is appliedto classify whether the lesion occurs or not. The efficiency andsensitivity even under small sample data is guaranteed.

B-1: Axial curvature eigenvalues of the anterior corneal surface arecalculated: a maximum curvature point and position coordinates in a datamatrix of an axial curvature of the anterior corneal surface arecalculated. An IS value of a difference between upper and lowerrefractive power at a position with a diameter of 6 mm is calculated. AnSRI and an SAI within a diameter of 4.5 mm are calculated.

B-2: Relative elevation eigenvalues of the anterior corneal surface arecalculated: a maximum elevation and position coordinates in a datamatrix of a relative elevation of the anterior corneal surface arecalculated.

B-3: Relative elevation eigenvalues of the posterior corneal surface arecalculated: a maximum elevation and position coordinates in a datamatrix of a relative elevation of the posterior corneal surface arecalculated.

B-4: Corneal thickness eigenvalues are calculated: a minimum thicknessand position coordinates in a data matrix of a corneal thickness arecalculated, and a thickness at a corneal vertex is calculated.

B-5: Distance eigenvalues are calculated: a distance from a position ofthe maximum elevation of the anterior corneal surface in step B-2 to aposition of the maximum elevation of the posterior corneal surface instep B-3 is calculated. A distance from the position of the maximumelevation of the anterior corneal surface in step B-2 to a position ofthe minimum corneal thickness in step B-4 is calculated. A distance fromthe position of the maximum elevation of the posterior corneal surfacein step B-3 to the position of the minimum corneal thickness in step B-4is calculated.

B-6: Corneal volume eigenvalues are calculated: volume integral isperformed on the data matrix of the corneal thickness within a radius of4.5 mm to obtain a corneal volume.

B-7: All of the eigenvalues in steps B-1 to B-6 are normalized, and allof the normalized case data eigenvalues are divided into a training setand a validation set according to a ratio of 7:3.

B-8: SVM is used for support vector model training. Feature training isperformed on feature data of the training set normalized in step B-7 bythe binary classification method using the SVM. A RBF kernel is selectedin the process, and optimal c and g are obtained to train data usingcross-validation and grid-search.

B-9: Model index evaluation is performed: predictions are made on thevalidation set, and then compared with a true value for evaluation tofinally obtain the sensitivity and specificity of the branch B method inidentifying the keratoconus.

B-10: Results are output: if test sensitivity and specificity of thetraining set in step B-9 meet requirements, probabilities of thekeratoconus of the branch B method for the certain case are recorded asp(Bl) and p(Br) respectively. Classification results are output:P(B)=p(Bl), (p(Bl)>p(Br)). p(Br), (p(Bl)<p(Br)).

The branch C method is as follows: this branch method takes a singlecase as a reference and upgrades the traditional Belin method. It notonly fully reflects the lesion features of the anterior and posteriorcorneal surfaces through highly enhanced data, but also improves thefeature dimension by changing different BFS values, so as to improve thespecificity of the statistical method and reduce the false alarm rate offalse positives.

C-1: Standard relative elevation data of the anterior corneal surface iscalculated: for the absolute elevation data of the anterior andposterior corneal surfaces, spherical fitting is performed on theabsolute elevation data of the anterior corneal surface within adiameter of 8 mm to obtain a BFS value, and an elevation differencebetween the data of the anterior corneal surface and the obtained BFS istaken as the standard relative elevation data of the anterior cornealsurface.

C-2: Feature elevation data of the anterior corneal surface iscalculated: for the absolute elevation data of the anterior andposterior corneal surfaces, data within a radius of 2 mm of a thinnestpoint is removed for spherical fitting by taking the absolute elevationdata of the anterior corneal surface within a diameter of 8 mm as abenchmark to obtain a BFS value. 5 groups of data are offset up and downrespectively by taking the current BFS as a benchmark and 0.2 mm as astride to obtain 11 groups of different BFS values. An elevationdifference between the data of the anterior corneal surface and theobtained different BFS is taken as the feature relative elevation dataof the anterior corneal surface.

C-3: Enhanced elevation data of the anterior corneal surface iscalculated: a difference is calculated between the standard relativeelevation data obtained in step C-1 and the 11 groups of featurerelative elevation data obtained in step C-2 to obtain 11 groups ofenhanced data of the anterior corneal surface.

C-4: Standard relative elevation data of the posterior corneal surfaceis calculated: for the absolute elevation data of the anterior andposterior corneal surfaces, spherical fitting is performed on theabsolute elevation data of the posterior corneal surface within adiameter of 8 mm to obtain a BFS value, and an elevation differencebetween the data of the posterior corneal surface and the obtained BFSis taken as the standard relative elevation data of the posteriorcorneal surface.

C-5: Feature elevation data of the posterior corneal surface iscalculated: for the absolute elevation data of the anterior andposterior corneal surfaces, data within a radius of 2 mm of a thinnestpoint is removed for spherical fitting by taking the absolute elevationdata of the posterior corneal surface within a diameter of 8 mm as abenchmark to obtain a BFS value. 5 groups of data are offset up and downrespectively by taking the current BFS as a benchmark and 0.2 mm as astride to obtain 11 groups of different BFS values. An elevationdifference between the data of the posterior corneal surface and theobtained different BFS is taken as the feature relative elevation dataof the posterior corneal surface.

C-6: Enhanced elevation data of the posterior corneal surface iscalculated: a difference is calculated between the standard relativeelevation data of the posterior corneal surface obtained in step C-4 andthe 11 groups of feature relative elevation data obtained in step C-5 toobtain 11 groups of enhanced data of the posterior corneal surface.

C-7: A critical threshold between the keratoconus cases and the normalcases in each group of data is counted in combination with all sampledata by taking a matrix of a total of 22 groups of enhanced data of theanterior and posterior corneal surfaces obtained in steps C-3 and C-6 asa feature.

C-8: Probabilities of the keratoconus of the branch C method for thecertain case are recorded as p(Cl) and p(Cr) respectively. p(Cl) andp(Cr) are obtained through accumulation by taking a difference betweeneach group of enhanced data calculated from a current case and thecritical threshold obtained in step C-7 as a weight ratio.Classification results are output: P(C)=p(Cl), (p(Cl)>p(Cr)). p(Cr),(p(Cl)<p(Cr)).

The branch D method is as follows: this branch method takes binocularcases as a reference, combines data of the four refractive maps for leftand right eyes, and reflects the features of the lesion itself byextracting the difference data of the topographies of both eyes, therebyimproving the identification accuracy of patients with the keratoconusas individuals.

D-1: Data orientations are unified: the data matrices of the fourrefractive maps for the right eye are mirrored in a longitudinaldirection, such that nasal and bitamporal orientations of the datamatrices of the four refractive maps for the left and right eyes areunified.

D-2: diff diagram matrices of the four refractive maps are obtained: apoint-to-point difference of the data matrices of the four refractivemaps for the left and right eyes is calculated respectively and then anabsolute value is taken to obtain the diff diagram data matrices of thefour refractive maps, as shown in FIG. 5 .

D-3: diff data features are calculated: an average value, a maximumvalue, and a standard deviation of all data in the diff diagram datamatrices of the four refractive maps within a diameter of 6 mm arecalculated respectively as feature quantities.

D-4: A critical threshold between the keratoconus cases and the normalcases in each group of data is counted in combination with all sampledata by taking the 12 groups of average values, maximum values, andstandard deviations of diff diagrams of the four refractive maps for thecorneas of the left and right eyes obtained in step D-3 as features, orfeatures of all types of diff data are normalized for training andtesting using the SVM classification method to give the optimalsensitivity and specificity.

D-5: A probability of the keratoconus of the branch D method for thecertain case is recorded as P(D). P(D) is obtained through accumulationby taking a difference between an eigenvalue of each group of diff datacalculated from a current case and the critical threshold obtained instep D-4 as a weight ratio.

5. Final results in the branch A, B, C, and D methods are weighted andobtained through accumulation to obtain a final probability of thekeratoconus of the certain caseP=w₁*P(A)+w₂*P(B)+w₃*P(C)+w₄*P(D)/(w₁+w₂+w₃+w₄). w₁, w₂, w₃, and w₄ arethe weights of the branch A, B, C, and D methods respectively. Theacquisition of the weights should make full use of each branch methodaccording to the target requirements, so as to achieve an optimalbalance of sensitivity and specificity, and strive for the smallestfalse negative rate and false positive rate while ensuring robustness.

The above-mentioned embodiments only express several implementations ofthe present disclosure, and the descriptions thereof are relativelyspecific and detailed, but they should not be thereby interpreted aslimiting the scope of the present disclosure. It should be noted thatthose of ordinary skill in the art can further make several variationsand improvements without departing from the idea of the presentdisclosure, but such variations and improvements shall all fall withinthe protection scope of the present disclosure. Therefore, theprotection scope of the present disclosure shall be subject to theappended claims.

What is claimed is:
 1. A method for early diagnosis of keratoconus based on multi-modal data, specifically comprising the following steps: 1) acquiring binocular multi-modal data comprising four refractive maps for corneas of both eyes and absolute corneal elevation data, wherein the four refractive maps for the corneas comprise an axial curvature map of an anterior corneal surface, a relative elevation topography of the anterior corneal surface, a relative elevation topography of a posterior corneal surface, and a corneal thickness topography; and the absolute corneal elevation data comprises absolute elevation data of the anterior and posterior corneal surfaces; 2) according to classified cases, associating the binocular multi-modal data with classification categories, and classifying the data according to requirements; 3) unifying data of topographies and the elevation data of the anterior and posterior corneal surfaces in the four refractive maps in step 2) into data matrices of a same size; 4) based on the above data, determining early keratoconus of the both eyes using four branch methods, which are respectively recorded as a branch A method, a branch B method, a branch C method, and a branch D method; wherein the branch A method is as follows: after data processing, sending all of the data matrices of the four refractive maps to a classification network of a deep convolutional network to identify sensitivity and specificity of the keratoconus and obtain a classification result P(A) output for a certain case; the branch B method is as follows: calculating eigenvalues of each graphic data matrix in all of the data matrices of the four refractive maps, and sending eigenvalue data to a binary classification method using a support vector machine (SVM) to identify the sensitivity and specificity of the keratoconus and obtain a classification result P(B) output for the certain case; the branch C method is as follows: comparing the absolute elevation data of the anterior and posterior corneal surfaces with best-fit-sphere (BFS) data to obtain a critical threshold between keratoconus cases and normal cases, so as to determine a classification result P(C) output for the certain case; and the branch D method is as follows: obtaining optimal sensitivity and specificity as well as a probability P(D) of the keratoconus of the certain case using the critical threshold or using an SVM classification method by taking an average value, a maximum value, and a standard deviation of the data matrices of the four refractive maps for left and right eyes as feature quantities; and 5) weighting and accumulating final results in the branch A method, the branch B method, the branch C method and the branch D method to obtain a final probability of the keratoconus of the certain case.
 2. The method for early diagnosis of keratoconus based on multi-modal data according to claim 1, wherein the branch A method comprises the following specific steps: A-1: performing data scaling: scaling all of the data matrices of the four refractive maps processed in step 3) to a size of 224×224 by linear interpolation; A-2: performing data normalization: dividing data in step A-1 into a training set and a validation set according to a ratio of 7:3, then calculating means and standard deviations of the data matrices of the four refractive maps on the training set respectively to correspondingly obtain 4 means and 4 standard deviations, and then normalizing data matrices of four refractive maps for all cases with the means and the standard deviations; A-3: based on classification network design of the deep convolutional network, performing binary classification on the data matrices of the four refractive maps using a Resnet50 classification network to identify a normal cornea and the keratoconus in one eye; A-4: training a classification model: connecting the data matrices of the four refractive maps according to a channel to obtain an input of 4 channels, wherein data amplification uses rotation, translation, and random fuzzy preprocessing, and a loss function uses a binary cross entropy function; using training weights of MobileNetV3 on an IMAGENET dataset as initial weights, and then performing fine-tuning training; and finally selecting a training weight with a smallest difference between loss values of the training set and the validation set as a training result; A-5: performing model index evaluation: making predictions on the validation set, and then comparing with a true value for evaluation to finally obtain the sensitivity and specificity of the branch A method in identifying the keratoconus; and A-6: outputting results: when test sensitivity and specificity of the training set in step A-5 meet requirements, recording probabilities of the keratoconus of the branch A method for the certain case as p(Al) and p(Ar) respectively; and outputting classification results: P(A)=p(Al), (p(Al)>p(Ar)), and p(Ar), (p(Al)<p(Ar)).
 3. The method for early diagnosis of keratoconus based on multi-modal data according to claim 1, wherein the branch B method comprises the following specific steps: B-1: calculating axial curvature eigenvalues of the anterior corneal surface: calculating a maximum curvature point and position coordinates in a data matrix of an axial curvature of the anterior corneal surface, calculating an inferior-superior (IS) value of a difference between upper and lower refractive power at a position with a diameter of 6 mm, and calculating a surface regularity index (SRI) and a surface asymmetry index (SM) within a diameter of 4.5 mm; B-2: calculating relative elevation eigenvalues of the anterior corneal surface: calculating a maximum elevation and position coordinates in a data matrix of a relative elevation of the anterior corneal surface; B-3: calculating relative elevation eigenvalues of the posterior corneal surface: calculating a maximum elevation and position coordinates in a data matrix of a relative elevation of the posterior corneal surface; B-4: calculating corneal thickness eigenvalues: calculating a minimum thickness and position coordinates in a data matrix of a corneal thickness, and calculating a thickness at a corneal vertex; B-5: calculating distance eigenvalues: calculating a distance from a position of the maximum elevation of the anterior corneal surface in step B-2 to a position of the maximum elevation of the posterior corneal surface in step B-3, calculating a distance from the position of the maximum elevation of the anterior corneal surface in step B-2 to a position of the minimum corneal thickness in step B-4, and calculating a distance from the position of the maximum elevation of the posterior corneal surface in step B-3 to the position of the minimum corneal thickness in step B-4; B-6: calculating corneal volume eigenvalues: performing volume integral on the data matrix of the corneal thickness within a radius of 4.5 mm to obtain a corneal volume; B-7: normalizing all of the eigenvalues in steps B-1 to B-6, and dividing all of the normalized case data eigenvalues into a training set and a validation set according to a ratio of 7:3; B-8: performing feature training on feature data of the training set normalized in step B-7 by the binary classification method using the SVM, wherein a radial basis function (RBF) kernel is selected in the process, and optimal c and g are obtained to train data using cross-validation and grid-search; B-9: performing model index evaluation: making predictions on the validation set, and then comparing with a true value for evaluation to finally obtain the sensitivity and specificity of the branch B method in identifying the keratoconus; and B-10: outputting results when test sensitivity and specificity of the training set in step B-9 meet requirements, recording probabilities of the keratoconus of the branch B method for the certain case as p(Bl) and p(Br) respectively; and outputting classification results: P(B)=p(Bl), (p(Bl)>p(Br)), and p(Br), (p(Bl)<p(Br)).
 4. The method for early diagnosis of keratoconus based on multi-modal data according to claim 1, wherein the branch C method comprises the following specific steps: C-1: calculating standard relative elevation data of the anterior corneal surface: for the absolute elevation data of the anterior and posterior corneal surfaces, performing spherical fitting on the absolute elevation data of the anterior corneal surface within a diameter of 8 mm to obtain a BFS value, and taking an elevation difference between the data of the anterior corneal surface and the obtained BFS as the standard relative elevation data of the anterior corneal surface; C-2: calculating feature elevation data of the anterior corneal surface: for the absolute elevation data of the anterior and posterior corneal surfaces, removing data within a radius of 2 mm of a thinnest point for spherical fitting by taking the absolute elevation data of the anterior corneal surface within a diameter of 8 mm as a benchmark to obtain a BFS value; and offsetting 5 groups of data up and down respectively by taking the current BFS as a benchmark and 0.2 mm as a stride to obtain 11 groups of different BFS values, and taking an elevation difference between the data of the anterior corneal surface and the obtained different BFS as the feature relative elevation data of the anterior corneal surface; C-3: calculating enhanced elevation data of the anterior corneal surface: calculating a difference between the standard relative elevation data obtained in step C-1 and the 11 groups of feature relative elevation data obtained in step C-2 to obtain 11 groups of enhanced data of the anterior corneal surface; C-4: calculating standard relative elevation data of the posterior corneal surface: for the absolute elevation data of the anterior and posterior corneal surfaces, performing spherical fitting on the absolute elevation data of the posterior corneal surface within a diameter of 8 mm to obtain a BFS value, and taking an elevation difference between the data of the posterior corneal surface and the obtained BFS as the standard relative elevation data of the posterior corneal surface; C-5: calculating feature elevation data of the posterior corneal surface: for the absolute elevation data of the anterior and posterior corneal surfaces, removing data within a radius of 2 mm of a thinnest point for spherical fitting by taking the absolute elevation data of the posterior corneal surface within a diameter of 8 mm as a benchmark to obtain a BFS value; and offsetting 5 groups of data up and down respectively by taking the current BFS as a benchmark and 0.2 mm as a stride to obtain 11 groups of different BFS values, and taking an elevation difference between the data of the posterior corneal surface and the obtained different BFS as the feature relative elevation data of the posterior corneal surface; C-6: calculating enhanced elevation data of the posterior corneal surface: calculating a difference between the standard relative elevation data of the posterior corneal surface obtained in step C-4and the 11 groups of feature relative elevation data obtained in step C-5 to obtain 11 groups of enhanced data of the posterior corneal surface; C-7: counting a critical threshold between the keratoconus cases and the normal cases in each group of data in combination with all sample data by taking a matrix of a total of 22 groups of enhanced data of the anterior and posterior corneal surfaces obtained in steps C-3 and C-6 as a feature; and C-8: recording probabilities of the keratoconus of the branch C method for the certain case as p(Cl) and p(Cr) respectively, wherein p(Cl) and p(Cr) are obtained through accumulation by taking a difference between each group of enhanced data calculated from a current case and the critical threshold obtained in step C-7 as a weight ratio; and outputting classification results: P(C)=p(Cl), (p(Cl)>p(Cr)), and p(Cr), (p(Cl)<p(Cr)).
 5. The method for early diagnosis of keratoconus based on multi-modal data according to claim 1, wherein the branch D method comprises the following specific steps: D-1: unifying data orientations: mirroring the data matrices of the four refractive maps for the right eye in a longitudinal direction, and unifying nasal and bitamporal orientations of the data matrices of the four refractive maps for the left and right eyes; D-2: obtaining diff diagram matrices of the four refractive maps: calculating a point-to-point difference of the data matrices of the four refractive maps for the left and right eyes respectively and then taking an absolute value to obtain the diff diagram data matrices of the four refractive maps; D-3: calculating diff data features: calculating an average value, a maximum value, and a standard deviation of all data in the diff diagram data matrices of the four refractive maps within a diameter of 6 mm respectively as feature quantities; D-4: counting a critical threshold between the keratoconus cases and the normal cases in each group of data in combination with all sample data by taking the 12 groups of average values, maximum values, and standard deviations of diff diagrams of the four refractive maps for the corneas of the left and right eyes obtained in step D-3 as features, or normalizing features of all types of diff data for training and testing using the SVM classification method to give the optimal sensitivity and specificity; and D-5: recording a probability of the keratoconus of the branch D method for the certain case as P(D), wherein P(D) is obtained through accumulation by taking a difference between an eigenvalue of each group of diff data calculated from a current case and the critical threshold obtained in step D-4 as a weight ratio. 